Configuration
Management

What is Configuration Management?

If we go by the definition, configuration management (or just "config management") is the systematic approach to organize, store, access, or basically maintaining all the settings of our backend app. The key word there is settings.

You can think of configuration as the DNA of your application. It decides how your code runs in different environments. The exact same application code can behave in completely different ways depending purely on its configuration — and that's by design.

The Common Misconception

When you say "configuration management" to most people, the first impression — the first idea that hits the mind — is storing things like:

• Database passwords and the secure connection URL of your database
• Secure authentication keys — secret keys for your JWT secrets, etc.
• API keys of external services — like your email delivery service

Those are the things that come to mind whenever we talk about config management. But thinking like that misses a lot of the scope.

The Full Scope of Config Management

Configuration management actually covers a wide range of application behaviors:

• How your application starts up — the bootstrap sequence and what it needs to begin running
• How it connects to external services — databases, caches, message queues, third-party APIs
• How it behaves in different environments — dev vs staging vs production behaviour
• What it logs, whether it logs, and where it logs — logging behaviour and destinations
• Where it sends metrics — performance metrics and business metrics destinations
• Which features are enabled or disabled — for this version of the deployment, and for which users

There is a lot of scope for config management. It's not just limited to your database URL or secret keys — it affects a lot of important behaviors of your backend application. That's exactly why it's such an important topic.

The E-Commerce Platform Example

Let's say you are building an e-commerce platform, and you're working on the backend. Your configuration might include a whole range of things, each with very different characteristics. Let's walk through every category that such a backend would need.

Breaking down each category in detail:

• Database connection details — timeout details, host, port, username, password, etc. All the connection details of your database.
• Payment processor API keys — if you're using Stripe, you'd have your Stripe API keys here.
• Feature flags — what features are enabled for this deployment and what's kept disabled. For example: you built a new checkout flow, and a feature flag controls whether you're still using the old checkout flow or the new one. You can even conditionally enable the new flow for some users and the old flow for others.
• Performance tuning parameters — like your connection pool size (the pool used for your database connections).
• Security settings — like session timeouts — whether it's 30 seconds, 60 seconds, etc.
• Business rules — for e-commerce, this might contain things like the maximum order amount allowed for a user.

Why These Configs Have Different Characteristics

Each of these configurations has fundamentally different characteristics — and recognising these differences is what drives every decision about how to store and protect them:

Some are sensitive and supposed to be kept secret — leaking them might cause a lot of damage and loss. Some are not secretive but they control how your application behaves while it's running. Some change frequently; others change once a month or once a quarter. Some stay the same across development, staging and production; others differ depending on the environment. This diversity is the core reason config management needs a real strategy.

The Distributed Systems Challenge

On top of all that diversity sits one big challenge: distributed systems. Most of our modern applications — all our modern backends — don't run in isolation anymore. Most of them are now part of a complex distributed system.

A distributed system consists of multiple moving parts:

• Multiple services
• Databases
• Caches (like Redis)
• Message queues
• Third-party integrations
• Authentication providers
• Email services
• ...and a lot more

All these different integration points — where you connect a separate service to your main backend — they all require configuration. Your backend obviously needs to know how to connect to these services. But on top of that, it also needs to know:

• How to handle failures — what happens when a service is down or slow
• How to optimize performance — depending on the environment and settings
• How to maintain security — depending on the environment and config sources

All of this depends on different environments, different settings, and different configs — which we can control from different sources. This is the reason that, because most modern backends run on distributed systems, configuration management becomes such an important part of your application.

And backend systems run in diverse environments — different cloud providers, on-premises servers, containers, serverless functions, edge functions — and all of these have their own requirements for different configurations. There's no single uniform target, which makes a systematic approach essential.

Configuration Chaos

If you don't have a systematic approach — if you don't have a dedicated pipeline, a dedicated strategy to manage your config — you will end up with what we call configuration chaos.

Configuration chaos means that instead of controlling configs from a centralized point, you have a mess. Here's what it looks like:

This last point is the killer: debugging your application becomes a nightmare because you cannot reproduce the issues. You don't really know what exactly happened, what particular config caused a particular issue, or what particular config caused a break in production. Centralized config management is what gives you that traceability.

Types of Configuration

Not all configurations are created equal. Understanding the different types of configuration data is very important for choosing the right storage mechanism, security measures, and access patterns — who can access what, and in which environments a particular config should be accessible. Let's go through each type.

5.1 — Application Settings

The most common kind of config you'll see in most backend apps. This includes:

• Log level — In development we set the log level to debug so we can see and differentiate between different logs while building the app locally. In production we change it to info so we don't clutter our valuable production logs.
• Port — Which port the backend runs on. Usually 8080 locally; in production (VPS or Kubernetes) you might have different ports.
• Connection pool size — The maximum pool size for your database connections. Backend apps use connection pooling to optimize DB connections.
• Timeout values — How long the server should wait for a request to complete.

5.2 — Database Configuration

Database config includes all the details your application needs to connect to your database:

• Host — where your database is running
• Port — the database port
• Username — database user
• Password — database password (sensitive!)
• Database name — the specific DB to connect to

All these parameters combined let you construct a connection URL, which your application uses at runtime to connect to the database. It might also have other parameters — like query timeouts (how long a database query should run before it times out).

5.3 — External Services

External services come in many types, and each has its own configuration needs — usually some kind of API key:

• Email providers — Mailchimp, Resend, etc. → you'll need an API key to initiate the client and send emails
• Payment processors — Stripe → a Stripe API key
• Authentication providers — Clerk → a Clerk API key

It doesn't matter which provider you use — you'll most probably have some kind of API key in your config that you need to cater to.

5.4 — Feature Flags

Feature flags exist so that you can dynamically enable or disable particular features of your application without changing code.

The classic example: you've built a new checkout flow (using a different API) and you don't want to roll it out to all users at once. You want to do A/B testing — only enable the new feature for a particular segment of users depending on their location. For instance, enable the new checkout for users from the US, but keep the old flow for users from India. Feature flags make this dynamic enabling/disabling possible — and they also need config management.

5.5 — Other Config Types

Beyond the four main categories, there are several more:

• Infra config — All your DevOps-related configurations.
• Security config — Your JWT secret, session secret, and anything related to security that affects your application.
• Performance tuning — If your backend is built with Go (Golang), you can set environment variables like the maximum number of CPUs (GOMAXPROCS), etc.
• Business rules — Different logic-related rules you want to enforce at the application level, but centralized in config rather than hard-coded.

Sources of Config (Storage)

With all these diverse config needs, we also have to think about where to store the configs. Where you store your configurations depends on a lot of parameters — security, speed, and your environment. Let's look at the main storage mechanisms.

6.1 — Environment Variables

The most common kind of config storage that most people have seen. If you've worked with backends before, you've encountered .env files. These are most common among Node.js apps, but they're also supported in Python, Go, and any language you build backends in.

How it works locally

In local environments, you have a file called .env. A very famous library called dotenv takes all your environment variables from the .env file and loads them into your operating system's environment.

How it works in deployment

If you're not local — say you're deploying in a containerized environment using Kubernetes — those technologies also support loading environment variables while deploying your application. Here's the typical cloud deployment workflow:

The deployment triggers; at a particular stage it fetches all your environment variables from a service — Vault (HashiCorp), Parameter Store, Azure, or Google Secret Manager (any cloud provider that offers secrets management) — and loads them into your environment. Then when your application starts, it reads those environment variables and functions accordingly. This is the most common storage mechanism for configurations.

6.2 — Config Files

The second kind of storage is files, which come in different formats:

One of the major disadvantages of JSON is that you cannot add comments — JSON has no comment support. That's why most people use YAML: you can add your configuration and add comments so it's easier for your team to share knowledge. TOML is a newer standard that's also popular.

6.3 — Key-Value Stores & Cloud Providers

Key-Value Stores

Beyond simple env values, there are dedicated cloud-native tools like Consul and etcd. Depending on your use case you can use any of them. Key-value stores are pretty lightweight and simple to use compared to more complicated, hierarchical data structures for config management. They act mostly like your environment variables.

Dedicated Cloud Secret Managers

Here the major players are:

• HashiCorp Vault
• AWS Parameter Store
• Azure Key Vault
• Google Secret Manager

These are dedicated services for config and secrets management that most companies use, because deployments today are distributed — Kubernetes, autoscaling, sometimes multiple cloud providers at once for scaling needs. Having all your configurations centralized in a tool like HashiCorp Vault — which already supports all these different environments, with dedicated docs and integrations — makes the integration as easy as possible. Using an external provider makes a lot of sense if you're running heavy user traffic in production.

Hybrid Strategies

Most of the time, teams actually use hybrid strategies. You have a loading phase where you construct your runtime application settings by fetching configs from multiple places, each with a priority:

For example, in the first priority you might fetch from AWS Parameter Store, then from a config.yaml, then from environment variables. You pre-decide which source has higher priority, and depending on the environment you load those configs conditionally. This gives you flexibility: secrets from a vault, structured settings from a file, and overrides from env vars — all merged into one runtime config.

Why Config Differs Per Environment

One natural question: why do we have different configs depending on the environment we're running in? The answer is simple — each environment has a set of priorities that the other environments don't need or shouldn't have.

Walking through each environment's priority:

• Development (local) — Running on localhost. Number one priority is developer productivity and debugging capabilities — so you can work faster and catch issues faster.
• Test — You trigger a GitHub Action that prepares a testing environment to run integration tests, unit tests, etc. Priority is automated validation and quality assurance.
• Staging — A hosted version of your application that functions as similar to production as possible, so you have a clear idea of how production will behave and can fix/debug issues accordingly. Priority is mirroring production.
• Production — The real thing. Priority is reliability, security, and performance.

Worked Example — Database Pool Size Across Environments

Take the database connection pool size as a concrete example of how the same setting differs per environment:

In production you set the pool size to 50 because you have to cater to a large user base and prepare for traffic spikes. In development, 10 works fine. The interesting one is staging: even though staging's main priority is to mirror production as much as possible to catch issues early, running a system that functions exactly like production would cost the same in cloud bills — and that's not something most teams want.

Configuration Security

These are fairly obvious points, but for the sake of covering the full scope, they're worth stating explicitly. There are five key security practices for config management.

8.1 — Never Hardcode Secrets

If you have secrets like your production database URL, the API key of your Clerk service, your email delivery service, or your payment processing service — you should never hardcode those secrets in your codebase. This is so obvious it barely needs explaining: hardcoded secrets end up in version control, get exposed in logs, and become permanent liabilities.

8.2 — Use a Cloud Secrets Manager

If possible, always go for a cloud secrets management service — HashiCorp Vault, AWS Parameter Store, Azure Key Vault, or Google Secret Manager. It's always a good idea to "over-engineer" when it comes to the security of your backend application.

These services already handle critical security features automatically:

• Encryption at rest — It doesn't matter where they store your config; whenever they store it, they encrypt it first, then store it in their database/storage.
• Encryption in transit — When you make an API call to fetch configs, they send the encrypted version. With your private key (stored in your infra, GitHub Action, repository secret, or Kubernetes environment), the configs are decoded and then used.

These are essential security features, and when you use a cloud provider like Vault or Parameter Store, all of this is already taken care of — you don't have to think about it.

8.3 — Access Control (Least Privilege)

If you have a fairly large developer team, take time to strategize who should have access to which configs, following the principle of least privilege:

• Frontend developers — should only have access to the configs they need: the backend API URL and API keys for frontend integrations.
• Backend engineers — should only have access to databases, Redis instances, Elasticsearch, etc.
• DevOps team only — configs for accessing cloud instances (e.g. EC2 instances) should be restricted to DevOps.

Following the least privilege principle, you strategize your access control so that each person only has access to what they actually need.

8.4 — Rotation

You should periodically rotate all your configs — API key secrets, JWT secrets, and similar sensitive values — so that they're not very prone to leakage. Regular rotation limits the damage window if a secret is ever compromised.

8.5 — Validation (The Most Important Point)

Most of the time, teams don't validate their environment variables or whatever source their configs are coming from. With dotenv, for example, you load all the variables into your environment and just access them via process.env.WHATEVER — with no checks.

Why does this matter so much? It's learned the hard way. If you have a mandatory environment variable requirement but fail to provide it, your production system either breaks or behaves in a strange way because it doesn't have access to a particular environment variable — and it's not easy to spot. Validating at startup turns a silent, mysterious production failure into a loud, immediate, obvious error message that tells you exactly which config is missing.

Code Examples

Below are practical implementations of config loading and validation in Go and Python — covering env vars, per-environment loading, hybrid sources, and startup validation.

9.1 — Config Loading & Validation in Go

package config

import (
    "fmt"
    "log"
    "os"
    "strconv"

    "github.com/go-playground/validator/v10"
    "github.com/joho/godotenv"
)

// Config holds all runtime application settings.
// The `validate` tags enforce rules at startup — this is the
// single most important safeguard for config management.
type Config struct {
    // Application settings
    Port     int    `validate:"required,min=1,max=65535"`
    LogLevel string `validate:"required,oneof=debug info warn error"`
    Env      string `validate:"required,oneof=development staging production"`

    // Database config (sensitive)
    DBHost     string `validate:"required"`
    DBPort     int    `validate:"required"`
    DBUser     string `validate:"required"`
    DBPassword string `validate:"required"`
    DBName     string `validate:"required"`
    DBPoolSize int    `validate:"required,min=1"`

    // External services (sensitive)
    StripeAPIKey string `validate:"required"`

    // Feature flags (optional, default false)
    NewCheckoutEnabled bool
}

// DatabaseURL constructs the connection URL from the parts.
func (c *Config) DatabaseURL() string {
    return fmt.Sprintf(
        "postgres://%s:%s@%s:%d/%s",
        c.DBUser, c.DBPassword, c.DBHost, c.DBPort, c.DBName,
    )
}

// Load reads env vars, applies defaults, then VALIDATES before returning.
// Called once at startup — fail loudly here, never silently in production.
func Load() (*Config, error) {
    // In local dev, load .env into the OS environment.
    // In production this is a no-op (vars already injected by the platform).
    _ = godotenv.Load()

    cfg := &Config{
        Port:               getEnvInt("PORT", 8080),        // default 8080
        LogLevel:           getEnv("LOG_LEVEL", "info"),    // default info
        Env:                getEnv("APP_ENV", "development"),
        DBHost:             os.Getenv("DB_HOST"),
        DBPort:             getEnvInt("DB_PORT", 5432),
        DBUser:             os.Getenv("DB_USER"),
        DBPassword:         os.Getenv("DB_PASSWORD"),
        DBName:             os.Getenv("DB_NAME"),
        DBPoolSize:         getEnvInt("DB_POOL_SIZE", 10),  // dev=10, prod=50
        StripeAPIKey:       os.Getenv("STRIPE_API_KEY"),
        NewCheckoutEnabled: getEnv("NEW_CHECKOUT", "false") == "true",
    }

    // THE critical step — validate everything before the app boots
    if err := validator.New().Struct(cfg); err != nil {
        return nil, fmt.Errorf("config validation failed: %w", err)
    }

    return cfg, nil
}

func getEnv(key, fallback string) string {
    if v := os.Getenv(key); v != "" {
        return v
    }
    return fallback
}

func getEnvInt(key string, fallback int) int {
    if v := os.Getenv(key); v != "" {
        if n, err := strconv.Atoi(v); err == nil {
            return n
        }
    }
    return fallback
}

// Usage in main.go:
//   cfg, err := config.Load()
//   if err != nil { log.Fatal(err) }  // crash early, crash loud

9.2 — Config Loading & Validation in Python

from enum import Enum
from functools import lru_cache

from pydantic import Field, computed_field
from pydantic_settings import BaseSettings, SettingsConfigDict


class Environment(str, Enum):
    DEVELOPMENT = "development"
    STAGING = "staging"
    PRODUCTION = "production"


class Settings(BaseSettings):
    """
    All runtime config. pydantic VALIDATES every field automatically
    when the object is constructed — mandatory fields without a default
    raise an error immediately at startup, not silently in production.
    """
    model_config = SettingsConfigDict(
        env_file=".env",        # load .env in local dev
        env_file_encoding="utf-8",
        case_sensitive=False,
    )

    # Application settings (with sensible defaults)
    port: int = Field(default=8080, ge=1, le=65535)
    log_level: str = Field(default="info", pattern="^(debug|info|warn|error)$")
    app_env: Environment = Environment.DEVELOPMENT

    # Database config — mandatory, no defaults (will fail if missing)
    db_host: str
    db_port: int = 5432
    db_user: str
    db_password: str          # sensitive — never hardcoded
    db_name: str
    db_pool_size: int = Field(default=10, ge=1)  # dev=10, prod=50, staging=2

    # External services — mandatory secret
    stripe_api_key: str

    # Feature flag — optional, defaults off
    new_checkout_enabled: bool = False

    @computed_field
    @property
    def database_url(self) -> str:
        return (
            f"postgresql://{self.db_user}:{self.db_password}"
            f"@{self.db_host}:{self.db_port}/{self.db_name}"
        )


@lru_cache  # load & validate once, reuse everywhere (singleton)
def get_settings() -> Settings:
    # Construction triggers validation. If a mandatory var is
    # missing, pydantic raises here — at startup, loudly.
    return Settings()


# Usage:
#   settings = get_settings()   # crashes early if config invalid
#   print(settings.database_url)
#   if settings.new_checkout_enabled: ...

9.3 — Example YAML Config File

For non-secret, structured settings, a YAML file (with comments, unlike JSON) is a common choice — as seen in real codebases like Authelia and Apache Answer:

# Server settings — note: YAML supports comments, JSON does not
server:
  port: 8080
  timeout_seconds: 60      # raise this for slow ops like AI image gen

log:
  level: debug             # debug in dev, info in production

database:
  host: localhost
  port: 5432
  name: ecommerce
  pool_size: 10            # dev=10, staging=2, prod=50
  query_timeout_seconds: 30

# Feature flags — dynamically enable/disable features
features:
  new_checkout_enabled: false
  rollout_regions: ["US"]   # A/B test: US gets new checkout

# Business rules — centralized, not hardcoded
business:
  max_order_amount: 500000

# NOTE: secrets (db password, Stripe key, JWT secret) do NOT
# belong here — those come from a secrets manager / env vars.

Further Reading & Documentation

Secrets Management Tools

Key-Value / Config Stores

Validation Libraries

Config File Formats & Libraries

MDN Web Docs